Depression is one of the most significant and prevalent mental health disorders in today’s world. Early detection of depression is critical, and this study aims to identify depression in individuals using information derived from social media. The use of social media for various purposes has grown in recent years, as these platforms provide valuable insights into both individuals and society. Social media can be effectively utilized to detect depression. Researchers have attempted to identify depression using various types of Data, such as images, text, and audio. Most studies have focused on using only one type of Data, such as text or images, for detection. While these methods have achieved notable results, they have limitations in accuracy that can be addressed by incorporating new methods and integrating multiple Data modalities into the model. In this study, we propose a multimodal model that analyzes text and images together to detect depression. Compared to similar models, our approach achieves an approximate 5% improvement in accuracy, reaching 89.87%, while utilizing significantly less of the original Dataset.

The objective of image Fusion for medical images is to combine multiple images obtained from various sources into a single image suitable for better diagnosis. Most of the state-of-the-art image fusing technique is based on nonfuzzy sets, and the fused image so obtained lags with complementary information. Intuitionistic fuzzy sets (IFS) are determined to be more suitable for civilian, and medical image processing as more uncertainties are considered compared with fuzzy set theory. In this paper, an algorithm for effectively fusing multimodal medical images is presented. In the proposed method, images are initially converted into Yager’ s intuitionistic fuzzy complement images (YIFCIs), and a new objective function called intuitionistic fuzzy entropy (IFE) is employed to obtain the optimum value of the parameter in membership and non-membership functions. Next, the YIFCIs are compared using contrast visibility (CV) to construct a decision map (DM). DM is refined with consistency verification to create a fused image. Simulations on several pairs of multimodal medical images are performed and compared with the existing Fusion methods, such as simple average, discrete cosine transform (DCT), redundant wavelet transform (RWT), intuitionistic fuzzy set, fuzzy transform and interval-valued intuitionistic fuzzy set (IVIFS). The superiority of the proposed method is presented and is justified. Fused image quality is also verified with various quality metrics, such as spatial frequency (SF), average gradient (AG), Fusion symmetry (FS), edge information preservation (QAB=F ), entropy (E) and computation time (CoT).

Introduction: Multimodal emotion recognition due to receiving information from different sensory resources (modalities) from a video has a lot of challenges and has attracted many researchers as a new method of human computer interaction. The purpose of this paper was to automatically recognize emotion from emotional speech and facial expression based on the neural mechanisms of the brain. Therefore, based on studies on brain-inspired models, a general framework for bimodal emotion recognition inspired by the functionality of the auditory and visual cortics and brain limbic system is presented. Methods: The hybrid and hierarchical proposed model consisted of two learning phases. The first step: the deep learning models for the representation of visual and auditory features, and the second step: a Mixture of Brain Emotional Learning (MoBEL) model, obtained from the previous stage, for Fusion of audio-visual information. For visual feature representation, 3D-convolutional neural network (3D-CNN) was used to learn the spatial relationship between pixels and the temporal relationship between the video frames. Also, for audio feature representation, the speech signal was first converted to the log Mel-spectrogram image and then fed to the CNN. Finally, the information obtained from the two above streams was given to the MoBEL neural network model to improve the efficiency of the emotional recognition system by considering the correlation between visual and auditory and Fusion of information at the feature level. Results: The accuracy rate of emotion recognition in video in the eNterface’ 05 Database using the proposed method was on average of 82%. Conclusion: The experimental results in the Database show that the performance of the proposed method is better than the hand-crafted feature extraction methods and other Fusion models in the emotion recognition.

In this study, we propose decision level Fusion of multimodal physiological signals to design an affect identification system using the MIT Database. Four types of physiological signals, including blood volume pressure (BVP), respiration rate (RSP), skin conductance and facial muscles activities (fEMG) were utilized as affective modalities. To collect the above-mentioned Database, researchers used personalized imagery to elicit the desired affective states from a single subject and recorded the corresponding physiological signals simultaneously. In this study, the best subset of features for each signal was determined using previously calculated time and frequency domain features. To this end, sequential floating forward selection (SFFS) and RELIEF feature selection algorithms were evaluated. A new feature set, formed by concatenating the selected features, was partitioned into three subsets. Each subset was then fed into a classifier to identify the desired affective states. The majority voting method was applied to fuse the results obtained by the subsystems. Three types of classification methods, namely SVM, LDA and KNN were evaluated to design an affect identification system. The results showed remarkable performance from the system in identifying the desired scenarios with an acceptable accuracy and speed of response. Using the RELIEF feature selection method, along with SVM as a classifier, an overall recognition accuracy of 93.8% was obtained, which is better than the results reported with the use of the above-mentioned Database so far.

Detecting and controlling stress levels in drivers is especially important to reduce the potential risks while driving. Accordingly, in this study, a detection system was presented to identify four levels of stress (low, neutral, high and very high) in drivers based on physiological signals. The proposed method used the drivedb Database, which includes the recording of physiological signals from 17 healthy volunteers while driving on specific routes on city streets and highways. A set of statistical and entropy features along with morphological features that were calculated only for the ECG signals, were used. The calculated features were applied as inputs to the classification units to detect stress levels. Support vector machine (SVM), k nearest neighbors (kNN) and decision tree (DT) were evaluated as classification methods. The main purpose of this study was to improve the accuracy of stress level detection using the idea of classifiers Fusion. To achieve this goal, the combination of individual classification units, each of which used only the features of one of the ECG, EMG and GSR signals, was performed by the Demster-Shafer method. Using genetic algorithm as feature selection method, SVM classifier and Dempster-Shafer Fusion strategy, the best stress detection accuracy of 96. 9% was obtained. While the highest detection accuracy among individual classifiers was 75% and obtained by a subsystem that used ECG features. The results show significant performance of the proposed method compared to previous studies that used the same Dataset.

A new concept named pseudo information measure is introduced. By this measure, Bayesian Fusion of independent sources of information is extended to a wide range of possible formulations and some new Fusion formulas are calculated. The coincidence between the performance of the proposed method of Fusion with the results that are expected by human logic and output sensitivity of the Fusion process are discussed. Also, we have discussed the resulting flexibility for map building applications. Map building by using the proposed Fusion formulas has been implemented on Khepera robot. The resulting map were fed to "A*" path planning algorithm for comparative purposes. For the resulting routes, two factors are considered: length and a danger measure which is an increasing function of the least distance of the path to obstacles. The results show that by using the proposed Fusion formulas, more informative maps of the environment are obtained by which more appropriate routes are achieved. Based on the selected function, there is a trade-off between the length of the resulting routes and their safety. This flexibility lets us choose the right Fusion function for different map building applications.